A wide variety of systems, such as systems used with audio and video media, need parameters optimized for the system's particular application. While the parameter optimization process is expected to be part of the design process and while the deployed design is expected to have been extensively tested in a wide range of scenarios, a class of applications exists in which parameter optimization can only be done after the system has been deployed, i.e., when operating in a real world application. In these situations, performance may depend on properties of the environment and these properties may be unknowable at design time and may even change as the system is being used. Different stakeholders may place or operate a particular system in unique statistical contexts that could not have been known by the original designers. A designer can make assumptions, but there may be no way to know when and if that assumption is valid.
One such example of this problem is found in audio watermarking technology used in broadcasting where the goal is to identify and count the number of listeners to a particular program as a means of evaluating the ad revenue that should be assigned to that program. There is no perfect watermarking system because one cannot simultaneously optimize all parameters. Consider the following property list: decodability of watermark, audibility of watermark, size of the information payload, response time to acquire the watermark codes, battery life if portable, the cost of decoders as measured in compute complexity, tolerance to signal degradation during transmission or encoding, dependency on the program material, and the acoustics of the listener's environment. Each application requires its own optimization and most applications are unique to the particulars of that user. Each broadcast station may well require its own optimization that by definition will deviate from some reference ideal that is a generic solution. The developers cannot do such optimization. Rather each station must tune to the properties of their own system. This means that each station must have a way of measuring the degree to which the current parameter set is or is not at an optimum.
But, in watermarking systems used in broadcasting to identify and count the number of listeners to a particular program, the broadcasted watermarked signal is received by listeners in their local environment, which may contain many other real world sound sources (e.g., the sound of engines, people talking, crowds, etc.) in addition to the broadcast. The local decoder, which has the responsibility of extracting the watermarking payload, is faced with the challenge of operating in an environment where both the program being transmitted and the sounds local to the decoder may undermine the performance of the decoder.
Even after the decoder receives and decodes the watermarked signal, the decoded payload for each listener is not immediately sent back to the radio station. The decoded payload is accumulated, perhaps once per day, and sent to the home office where an additional set of rules is applied to approximate the number of real listeners, the ratings. The final reports of listeners are eventually distributed back to the subscriber stakeholders, which may include advertising agencies, station sales staff, and others. This process can take days and it may take weeks for the ratings reports to be available. Significant delays are introduced.
Because of the long delay, optimization based on ratings is extremely difficult. Consider that a broadcaster sets a particular parameter to 17, and then a week later after getting a report, sets the parameter to 19 to see if that change influences the statistics. During the delay, a lot may have changed: school vacation may change the available listeners, a snow storm shuts down the number of drivers commuting to work, a world-series sport events takes place, the program director changes the type of audio being broadcast, and so on. In other words, if the change from 17 to 19 made a difference, there is no way to know if that change was produced by unrelated events or if the change resulted from the new parameter value. The statistics are time varying and arise from numerous unknown events. Stations may speculate, but they have no way to know if their guesses are relevant or accurate. Because the process of measuring listeners requires averaging to reduce data noise, and because averaging requires a long time span, the other variables also influence the result.